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Abstract 

We examine carefully the rationale underlying the approaches to belief change 
taken in the literature, and highlight what we view as methodological problems. 
We argue that to study belief change carefully, we must be quite explicit about 
the "ontology" or scenario underlying the belief change process. This is something 
that has been missing in previous work, with its focus on postulates. Our analysis 
shows that we must pay particular attention to two issues that have often been 
taken for granted: The first is how we model the agent's epistemic state. (Do we 
use a set of beliefs, or a richer structure, such as an ordering on worlds? And if 
we use a set of beliefs, in what language are these beliefs are expressed?) We show 
that even postulates that have been called "beyond controversy" are unreasonable 
when the agent's beliefs include beliefs about her own epistemic state as well as the 
external world. The second is the status of observations. (Are observations known 
to be true, or just believed? In the latter case, how firm is the belief?) Issues 
regarding the status of observations arise particularly when we consider iterated 
belief revision, and we must confront the possibility of revising by (p and then by 
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1 Introduction 



The problem of belief change — how an agent should revise her beliefs upon learning 
new information — has been an active area of research in both philosophy and artificial 
intelligence. The problem is a fascinating one in part because it is clearly no unique 
answer. Nevertheless, there is a strong intuition that one wants to make minimal changes, 
and all the approaches to belief change in the literature, such as ||Alchourr6n, Gardenfors, 



and Makinson 1985| ; pardenfors 1988| ; [Katsuno and Mendelzon 1991a|l , try to incorporate 
this principle. However, approaches differ on what constitutes a minimal change. This 
issue has come to the fore with the spate of recent work on iterated belief revision (see. 



for example. 


Boutilier 1996 




Boutilier and Goldszmidt 1993 




Darwiche and Pearl 1994 


Freund and Lehmann 1994 


; Lehmann 1995|; |Levi 1988; Wilhams 1994||). 



The approaches to belief change typically start with a collection of postulates, argue 
that they are reasonable, and prove some consequences of these postulates. Often a 
semantic model for the postulates is provided and a representation theorem is proved 
(of the form that every semantic model corresponds to some belief revision process, and 
that every belief revision process can be captured by some semantic model). Our goal 
here is not to introduce yet another model of belief change, but to examine carefully the 
rationale underlying the approaches in the literature. The main message of the paper is 
that describing postulates and proving a representation theorem is not enough. While it 
may have been reasonable when research on belief change started in the early 1980s to just 
consider the implications of a number of seemingly reasonable postulates, it is our view 
that it should no longer be acceptable just to write down postulates and give short English 
justifications for them. While postulates do provide insight and guidance, it is also 
important to describe what, for want of a better word, we call the underlying ontology or 
scenario for the belief change process. Roughly speaking, this means describing carefully 
what it means for something to be believed by an agent and what the status is of new 
information that is received by the agent. This point will hopefully become clearer as 
we present our critique. We remark that even though the issue of ontology is tacitly 
acknowledged in a number of papers (for example, in the last paragraph of ||Lehmann| 
1995|| ), it rarely enters into the discussion in a significant way.Q We hope to show that 



ontology must play a central role in all discussions of belief revision. 

Our focus is on approaches that take as their starting point the postulates for belief 
revision proposed by Alchourron, Gardenfors, and Makinson (AGM from now on) ||198 5 



but our critique certainly applies to other approaches as well, in particular, Katsuno 
and Mendelzon's belief update ||1991b|| ; see Section]^. The AGM approach assumes that 
an agent's epistemic state is represented by a belief set, that is, a set K of formulas in 
a logical language Cf\ What the agent learns is assumed to be characterized by some 



recent manuscript by Hansson |1998a] does raise some issues of ontology in the context of justifi- 
cation of beliefs, but no particular ontology for belief revision is discussed. 

^For example, Gardenfors [1988, p. 21] says "A simple way of modeling the epistemic state of an 
individual is to represent it by a set of sentences." 
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formula v?, also in £; K * ip describes the belief set of an agent that starts with belief set 
K and learns (f^ 

There are two assumptions implicit in this notation: 



• The functional form of * suggests that all that matters regarding how an agent 
revises her beliefs is the belief set and what is learnt. 

• The notation suggests that the second argument of * can be an arbitrary formula 
in C. But what does it mean to revise by false? In what sense can false be learnt? 
More generally, is it reasonable to assume that an arbitrary formula can be learnt 
in a given epistemic state? 



The first assumption is particularly problematic when we consider the postulates 
that AGM require * to satisfy. These essentially state that the agent is consistent in 
her choices, in the sense that she acts as though she has an ordering on the strength 
IGardenfors and Makinson 19881; Crove 1988] 



of her beliefs 



or an ordering on possi- 



ble worlds IPoutilier 1994| ; [Grove 1988| ; [Katsuno and Mendelzon 1991b|] , or some other 



predetermined manner of choosing among competing beliefs ||Alchourr6n, Gardenfors, 



[and Makinson 1985|| . However, the fact that an agent's epistemic state is characterized 
by a collection of formulas means that the epistemic state cannot include information 
about relative strength of beliefs (as required for the approach of, say, fGardenfors and 
[Makinson 1988|| ), unless this information is expressible in the language. Note that if C 
is propositional logic or first-order logic, such information cannot be expressed. On the 
other hand, if C contains conditional formulas of the form p > q, interpreted as "if p is 
learnt, then q will be believed", then constraints on the relative strength of beliefs can 
be expressed (indirectly, by describing which beliefs will be retained after a revision). 

Problems arise when the language is not rich enough to express relative degrees of 
strength in beliefs. Consider, for example, a situation where K = Cl{p A q) (the logical 
closure of p A q; that is, the agent's beliefs are characterized by the formula p A q and 
its logical consequences), and then the agent learns ip = -ip\/ -iq. We can imagine that 
an agent whose belief in p is stronger than her belief in q would have K * p = Cl{p). 
That is, the agent gives up her belief in q, but retains a belief in p. On the other hand, if 
the agent's belief in q is stronger than her belief in p, it seems reasonable to expect that 
K * (f = Cl{q). This suggests that it is unreasonable to take * to be a function if the 
representation language is not rich enough to express what may be significant details of 
an agent's epistemic state. 

We could, of course, assume that information about the relative strength of beliefs 
in various propositions is implicit in the choice of the revision operator *, even if it is 

■^One of the reviewers considered the usage of the terms learns and observes as introducing a bias 
regarding the nature of the behef revision process. We feel that this comment highUghts the need for a 
more concrete ontology for belief revision. We continue to use these terms, since they are quite standard 
in the literature, but we admit that they are indeed biased. There may well be ontologies for which they 
are inappropriate. 
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not contained in the language. This is perfectly reasonable, and also makes it more 
reasonable that * be a function. However, note that we can then no longer assume 
that we use the same * when doing iterated revision, since there is no reason to believe 
that the relative strength of beliefs is maintained after we learn a formula. In fact, in a 
number of recent papers [[Boutilier 1996| ; [Boutilier and Goldszmidt 1993| ; [briedman and 
[Halpern 1999 ; Nayak 1994| ; ^pohn 1988 ; Williams 1994 |, * is defined as a function from 
(epistemic states x formulas) to epistemic states, but the epistemic states are no longer 
just belief sets; they include information regarding relative strengths of beliefs. The 
revision function on epistemic states induces a mapping from (belief sets x formulas) to 
belief sets, but at the level of belief sets, the mapping may not be functional; for a belief 
set K and formula if, the belief set K * (f may depend on what epistemic state induced 
K. Thus, the effect of * on belief sets may change over time.0 

There is certainly no agreement on what postulates belief change should satisfy. How- 
ever, the following two postulates are almost universal: 



(f ^ K * ip 

if K is consistent and ip & K, then K * p = K. 



These postulates have been characterized by Rott | [L989| j as being "beyond controversy" . 
Nevertheless, we argue that they are not as innocent as they may at first appear. 

The first postulate says that the agent believes the last thing she learns. Making sense 
of this requires some discussion of the underlying ontology. It certainly makes sense if 
we take (as Gardenfors ||1988|| does) the belief set K to consist of all formulas that are 
accepted by the agent (where "accepted" means "treated as true"), and the agent revises 
by ip only if ip has somehow come to be accepted. However, note that deciding when a 
formula has come to be accepted is nontrivial. In particular, just observing ip will not in 
general be enough for accepting ip. Acceptance has a complex interaction with what is 
already believed. For example, imagine a scientist who believes that heavy objects drop 
faster than light ones, climbs the tower of Pisa, drops a 5 kilogram textbook and a 500 
milligram novel, and observes they hit the ground at the same time. This scientist will 
probably not accept that the time for an object to fall to the ground is independent of 
its weight, on the basis of this one experiment (although perhaps repeated experiments 
may lead her to accept it). 

While the acceptance point of view is certainly not unreasonable, the fact that just 
observing ip is not necessarily enough for acceptance often seems forgotten. It also seems 
hard to believe that false would ever be accepted. More generally, it is far from obvious 
that in a given epistemic state K we should allow arbitrary consistent formulas to be 
accepted. Intuitively, this does not allow for the possibility that some beliefs are held 



"^Freund and Lehmann |1994| have called the viewpoint that * may change over time the dynamic 
point of view. However, this seems somewhat of a misnomer when applied to papers such as | Boutilier 



1996; Boutilier and Goldszmidt 1993; Friedman and Halpern 1999; Williams 1994 1, since there * in fact 



is static, when viewed as a function on epistemic states and formulas. 
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so firmly that their negations could never be accepted. (Later we describe an ontology 
where observations are taken to be known in which in fact some consistent formulas will 
not be accepted in some epistemic states.) 

While the second postulate is perhaps plausible if we cannot talk about epistemic 
importance or strength of belief in the language, it is less so once we can talk about 
such things (or if either epistemic importance or strength of belief is encoded in the 
epistemic state some other way). For suppose that (p & K. Why should K * ip = Kl 
It could well be that being informed of raises the importance of if in the epistemic 
ordering, or the agent's strength of belief in 99. If strength of belief can be talked about 
in the language, then a notion of minimal change should still allow strengths of belief 
to change, even when something expected is observed. Even if we cannot talk about 
strength of belief in the language, this observation has an impact on iterated revisions. 
For example, one assumption made by Lehmann ||1995|| (his postulate 14) is that if p is 
believed after revising by then revising by [(p-p-ip] — that is, revising by 99 then p then 
if) — is equivalent to revising by [y^- ■?/']■ But consider a situation where after revising by 
the agent believes both p and g, but her belief in q is stronger than her belief in p. We 
can well imagine that after learning -ip V -ig in this situation, she would believe -ip and 
q. However, if she first learned p and then ^p V -ig, she would beheve p and -ig, because, 
as a result of learning p, she would give p higher epistemic importance than g. In this 
case, we would not have - p ■ (~ip V -ig)] = [(/? • (-ip V ~ig)]. In light of this discussion, it 
is not surprising that the combination of the second postulate with a language that can 
talk about epistemic ordering leads to technical problems such as Gardenfors' triviality 
result pH^] . 

To give a sense of our concerns here, we discuss two basic ontologies. The first 
ontology that seems (to us) reasonable assumes that the agent has some knowledge as 
well as beliefs. We can think of the formulas that the agent knows as having the highest 
state of epistemic importance. In keeping with the standard interpretation of knowledge, 
we also assume that the formulas that the agent knows are true in the world. Since agents 
typically do not have certain knowledge of very many facts, we assume that the knowledge 
is augmented by beliefs (which can be thought of as defeasible guides to action). Thus, 
the set of formulas that are known form a subset of the belief set. We assume that 
the agent observes the world using reliable sensors; thus, if the agent observes then 
the agent is assumed to know After observing the agent adds to his stock of 
knowledge, and may revise his belief set. Since the agent's observations are taken to be 
knowledge, the agent will believe ip after observing ip. However, the agent's epistemic 
state may change even if she observes a formula that she previously believed to be true. 
In particular, if the formula observed was beheved to be true but not known to be true. 



^In fact, the phrase "rehable sensors" is somewhat too strong. We can deal with unreHable sensors 
in our framework by exphcitly modehng the difference between the sensor reading the agent observes, 
which we take to be a rehable observation, and the actual state of the external w orld that the sensor 
is, unreliably, measuring or reporting. See [Boutilier, Friedman, and Halpern 1998 1 for a more detailed 
discussion of this point. 
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after the observation it is known. Note that, in this ontology, the agent never observes 
false, since false is not true of the world. In fact, the agent never observes anything that 
contradicts her knowledge. Thus, K*y:> is defined only for formulas (p that are compatible 
with the agent's knowledge. Moving to iterated revision, this means we cannot have a 
revision by (p followed by a revision by -up. This ontology underlies some of our earlier 
work | 1997 , |1997 , 1999 |. As we show here, by taking a variant of Darwiche and Pearl's 
approach | |1994| |, we can also capture this ontology. This ontology captures the essence 
of Bayesian updating in probabilistic reasoning. By taking observations to be known, we 
are essentially giving observed events probability 1 and conditioning on them. 

We can consider a second ontology that has a different flavor. In this ontology, if we 
observe something, we believe it to be true and perhaps even assign it a strength of belief. 
But this assignment does not represent the strength of belief of the observation in the 
resulting epistemic state. Rather, the belief in the observation must "compete" against 
current beliefs if it is inconsistent with these beliefs. In this ontology, it is not necessarily 
the case that (p E K * ip, just as it is not the case that a scientist will necessarily adopt 
the consequences of his most recent observation into his stock of beliefs (at least, not 
without doing some additional experimentation). Of course, to flesh out this ontology, 
we need to describe how to combine a given strength of belief in the observation with 
the strengths of the beliefs in the original epistemic state. Perhaps the closest parallel 
in the uncertainty literature is something like the Dempster-Shafer rule of combination 



Shafer 1976|| , which gives a rule for combining two separate bodies of belief. We believe 



that this type of ontology deserves further study. We sketch one particular approach to 
modeling this ontology in Section ^ and refer the reader to poutilier, Friedman, and 
Palpern 1998|| for a more thorough treatment of it. A quite different treatment of this 
problem, more in the spirit of the AGM approach, has been studied by Hansson ||1991 

[Hansson 1998b|| for an overview of this work. 



Makinson [19971, and others; see 



The rest of the paper is organized as follows. In Section ^, we review the AGM 
framework, and point out some problems with it. In Section |^, we consider proposals 
for belief change and iterated belief change from the literature due to Boutilier ||1996|| , 
Darwiche and Pearl ||1994|| , Freund and Lehmann ||1994]| , and Lehmann ||1995|| , and try to 
understand the ontology implicit in the proposal (to the extent that one can be discerned). 
In Section ^, we consider the first ontology discussed above in more detail. We conclude 
with some discussion in Section |^. 



2 AGM Belief Revision 

In this section we review the AGM approach to belief revision. As we said earlier, this 
approach assumes that beliefs and observations are expressed in some language C It is 
assumed that £ is closed under negation and conjunction, and comes equipped with a 
compact consequence relation l-£ that contains the propositional calculus and satisfies 
the deduction theorem. The agent's epistemic state is represented by a belief set, that 
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is, a set of formulas in £ closed under deduction. There is also assumed to be a revision 
operator * that takes a belief set K and a formula (p and returns a new belief set K * (p, 
intuitively, the result of revising K by (f. The following AGM postulates are an attempt 
to characterize the intuition of "minimal change" : 

Rl. K * if is a belief set 

R2. ip e K *(p 

R3. K*ipC Cl{KU{ip}) 

R4. li^if^K then Cl{K U {cp}) C K * cp 

R5. K * if = Cl{false) if and only if l-£ -up 

R6. U \-c ^ i' then K * (p = K * ip 

R7. K*{ipAip)C Cl{K * U {V^}) 

R8. \i^i)^K*(p then C/(/s: * U {^/'}) C fsT * ((^ A ^Z-) 

The essence of these postulates is the following. Revising K hy ip gives a belief set 
(Postulate Rl) that includes (p (R2). If (p is consistent with then K * (p consist 
precisely of those beliefs implied by the combination of the old beliefs with the new belief 
(R3 and R4). Note that it follows from R1-R4 that if^p e K, then K = K*ip. The next 
two postulates discuss the coherence of beliefs. R5 states that as long as (p is consistent, 
then so is * and R6 states that the syntactic form of the new belief does not affect 
the revision process. The last two postulates enforce a certain coherency on the outcome 
of revisions by related beliefs. Basically they state that if ip is consistent with K *ip then 
K * {(p Alp) is the result of adding ip to K * ip. 

The intuitions described by AGM is based on one-step (noniterated) revision. Nev- 
ertheless, the AGM postulates do impose some restrictions on iterated revisions. For 
example, suppose that q is consistent with K * p. Then, according to R2 and R3, 
{K*p)*q = C\{K*pU{q}). Using R7 and R8 we can conclude that {K*p)*q = K*{pAq). 

There are several representation theorems for AGM belief revision; perhaps the easiest 



to understand is due to Grove ||1988|| . We discuss a slight modification, due to Boutilier 



1994|| and Katsuno and Mendelzon | ]1991b| |: Let an C-world be a complete and consistent 



truth assignment to the formulas in C. Let W consist of all the £-worlds, and let ^ be 
a ranking^ that is, a total preorder, on the worlds in W. Let min^ consist of all the 
minimal worlds with respect to ^, that is, all the worlds w such that there is no w' with 
w' -< w. With ^ we can associate a belief set Bel(^), consisting of all formulas (p that 
are true in all the worlds in min^. Moreover, we can define a revision operator *, by 
taking Bel(^) * (p to consist of all formulas ip that are true in all the minimal yj-worlds 
according to ^. It can be shown that * satisfies the AGM postulates (when its first 
argument is Bel(^)). Thus, we can define a revision operator by taking a collection of 
orderings :<k, one for each belief set K. To define K * ip for a belief set K, we apply 
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the procedure above, starting with the ranking corresponding to K.f\ Furthermore, 
Grove ||1988|| , Katsuno and Mendelzon ||1991b|] , and Boutiher |p.994|| show that every behef 



revision operator satisfying the AGM axioms can be characterized in this way. 

This elegant representation theorem also brings out some of the problems with the 
AGM postulates. First, note that a given revision operator * is represented by a family of 
rankings, one for each belief set. There is no necessary connection between the rankings 
corresponding to different belief sets. It might seem more reasonable to have a more 
global setting (perhaps one global ranking) from which each element in the family of 
rankings arises. 

A second important point is that the epistemic state here is a function not only of 
the belief set, but also of the ranking. The latter, however, is represented only implicitly. 
Each ranking ^ is associated with a belief set Bel(^), but it is the ranking that gives 
the information required to describe how revision is carried out. The belief set does not 
suffice to determine the revision; there are many rankings ^ for which the associated 
belief set Bel(^) is K. Since the revision process only gives us the revised belief set, not 
the revised ranking, the representation does not support iterated revision. 

This suggests that we should consider, not how to revise belief sets, but how to revise 
rankings. More generally, whatever we take to be our representation of the epistemic 
state, it seems appropriate to consider how these representations should be revised. We 
can define an analogue of the AGM postulates for epistemic states in a straightforward 
way (cf. [[I'riedman and Halpern 1994|| ): Taking E to range over epistemic states and 



Bel{E) to represent the belief set associated with epistemic state E, we have 

Rl'. E * (f is an epistemic state 

R2'. ^ G Bel{E * ^) 

R3'. Bel{E *ip)C Cl{Bel{E) U {ip}) 

and so on, with the obvious syntactic transformation. In fact, as we shall see in the next 
section, a number of processes for revising epistemic states have been considered in the 
literature, and in fact they all do satisfy these modified postulates. 

Finally, even if we restrict attention to belief sets, we can consider what happens if 
the underlying language C is rich enough to talk about how revision should be carried 
out. For example, suppose C includes conditional formulas, and we want to find some 
ranking ^ for which the corresponding belief set is K. Not just any ranking ^ such 
that Bel(^) = K will do here. The beliefs in K put some constraints on the ranking. 
For example, if p > g is in if and p ^ K, then the minimal ^-worlds satisfying p must 
all satisfy q, since after p is learnt, q is believed. Once we restrict to rankings :<k that 
are consistent with K then the AGM postulates are no longer sound. This point has 
essentially been made before ||Boutilier 1992| ; pi.ott 1989|] . However, it is worth stressing 



^In this construction, for each behef set K other than the inconsistent behef set, we have Bel{<K) ~ 
K. The inconsistent behef set gets special treatment here. 
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the sensitivity of the AGM postulates to the underlying language and, more generally, 
to the choice of epistemic state. 



3 Proposals for Iterated Revision 

We now briefly review some of the previous proposals for iterated belief change, and point 
out how the impact of the observations we have been making on the approaches. Most 
of these approaches start with the AGM postulates, and augment them to get seemingly 
appropriate restrictions on iterated revision. This is not an exhaustive review of the 
literature on iterated belief revision by any stretch of the imagination. Rather, we have 
chosen a few representative approaches that allow us to bring out our methodological 
concerns. 



3.1 Boutilier's Approach 

As we said in the previous section, Boutilier takes the agent's epistemic state to consist 



of a ranking of possible worlds. Boutilier ||1996|| describes a particular revision operator 



*B on epistemic states. This revision operator maps a ranking ^ of possible worlds and 
an observation to a revised ranking :< *By^ such that (a) ^ *b'P satisfies the conditions 
of the representation theorem described above, that is, the minimal worlds in ^ *b<^ are 
precisely the minimal yj-worlds in ^, and (b) in a precise sense, ^ *_bv^ is the result of 
making the minimal number of changes to ^ required to guarantee that all the minimal 
worlds in ^ *B^p satisfy (p. Given a ranking ^ and a formula the ranking ^ *b^p is 
identical to ^ except that the minimal yj-worlds according to ^ have the minimal rank 
in the revised ranking, while the relative ranks of all other worlds remains unchanged. 

Boutilier characterizes the properties of his approach as follows. Suppose that, start- 
ing in some epistemic state, we revise by </?!,..., Further suppose (pi+i is consis- 
tent with the beliefs after revising by </?!,..., y^j. Then the beliefs after revising by 
ipi, . . . ,ipn are precisely the beliefs after observing ipi A . . . A ipn- (More precisely, given 
any ranking ^, the belief set associated with the ranking ^ *b^i*b ■ ■ ■ *b 'Pn is the 
same as that associated with the ranking :< *b {'Pi A ... A ipn) . Note, however, that 
:< *B'fi *B ■ ■ ■ *B fn *b(v^ A . . . A (fn) in general.) Thus, as long as the agent's 
new observations are not surprising, the agent's beliefs are exactly the ones she would 
have had had she observed the conjunction of all the observations. This is an immediate 
consequence of the AGM postulates, and thus holds for any approach that attempts to 
extend the AGM postulates to iterated revision. 

What happens when the agent observes a formula (pn+i that is inconsistent with 
her current beliefs? Boutilier shows that in this case the new observation nullifies the 
impact of the all the observations starting with the most recent one that is inconsis- 
tent with ipn+i- More precisely, suppose yjj+i is consistent with the belief after observ- 
ing ipi,...,(pi for all i < n, but (fn+i is inconsistent with the beliefs after observing 
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yji, . . . ,<^„. Let k be the maximal index such that (fn+i is consistent with the behefs 
after learning ipi, . . . ^ip^. The agent's beliefs after observing (^n+i are the same as her 
beliefs after observing ipi, . . . ^Lpk^ipn+i- Thus, the agent acts as though she did not 
observe v^fc+i, 

Boutilier does not provide any argument for the reasonableness of this ontology. In 
fact, Boutilier's presentation (like almost all others in the literature) is not in terms of 
an ontology at all; he presents his approach as an attempt to minimize changes to the 
ranking. While the intuition of minimizing changes to the ranking seems reasonable 
at first, it becomes less reasonable when we realize its ontological implications. The 
following example, due to Darwiche and Pearl ||1994|| , emphasizes this point. Suppose we 
encounter a strange new animal and it appears to be a bird, so we believe it is a bird. 
On closer inspection, we see that it is red, so we believe that it is a red bird. However, 
an expert then informs us that it is not a bird, but a mammal. Applying Boutilier's 
revision operator, we would no longer believe that the animal is red. This does not seem 
so reasonable. 

One more point is worth observing: As described by Boutilier ||1996||, his approach 



does not allow revision by false. While we could, of course, modify the definition to handle 
false, it is more natural simply to disallow it. This suggests that, whatever ontology is 
used to justify Boutilier's approach, in that ontology, revising by false should not make 
sense. 



3.2 Freund and Lehmann's Approach 

Freund and Lehmann [p.994]| stick close to the original AGM approach. They work with 
belief sets, not more general epistemic states. However, they are interested in iterated 
revision. They consider the effect of adding just one more postulate to the basic AGM 
postulates, namely 



FL. If G K, then K * (p = K_i_ * (p, 



where K± is the inconsistent belief set, which consists of all formulas. 

Suppose * satisfies R1-R8 and FL. Just as with Boutilier's approach, if yjj+i is con- 
sistent with the beliefs after learning ipi, . . . ,ipi for i < n — 1, then K * ipi * . . . * = 
K * {{fi A ... A ifn)- However, if we then observe v'n+i, and it is inconsistent with 
K * ifi /\ . . . /\ipn, then K *ipi* . . .* (pn+i = K± * (pn+i- That is, observing something in- 
consistent causes us to retain none of our previous beliefs, but to start over from scratch. 
While the ontology here is quite simple to explain, as Freund and Lehmann themselves 
admit, it is a rather severe form of belief revision. Darwiche and Pearl's red bird example 
applies to this approach as well. 



9 



3.3 Darwiche and Pearl's Approach 



Darwiche and Pearl ||1994|| suggest a set of postulates extending the AGM postulates, 



and claim to provide a semantics that satisfies them. Their intuition is that the revision 
operator should retain as much as possible certain parts of the ordering among worlds 
in the ranking. In particular, if w and w' both satisfy y?, then a revision by should 
not change the relative rank of w and w' . Similarly, if both w and w' satisfy -k/?, then 
a revision should not change their relative rank. They describe four postulates that are 
meant to embody these intuitions: 

CI. \i Lp\- ip, then {K *tp)*(p = K*(p 

C2. If (y9 I — then {K *^)*(f = K*Lp 

C3. If ijj E K * (f, then ip E (K * ^) * Lp 

C4. If -^-ip ^ K * Lf, then -^tp ^ {K * ip) * Lp 



Freund and Lehmann ||1994|| point out that C2 is inconsistent with the AGM postu- 
lates. This observation seems inconsistent with the fact that Darwiche and Pearl claim to 
provide an example of a revision method that is consistent with their postulates. What 
is going on here? It turns out that the issues raised earlier help clarify the situation. 

The semantics that Darwiche and Pearl use as an example is based on a special case of 



Spohn's ordinal conditional functions (OCFs) [ 1988|] called n-rankings [ Goldszmidt and 



[Pearl 19T?2| . A K-ranking associates with each world either a natural number n or cxd. 



with the requirement that for at least one world wq, we have k,{wq) = 0. We can think 
of n{w) as the rank of w, or as denoting how surprising it would be to discover that w 
is the actual world. If = 0, then world w is unsurprising; if k{w) = 1, then w is 

somewhat surprising; if k,{w) = 2, then w is more surprising, and so on. If k{w) = oo, 
then w is impossible.[] OCFs provide a way of ranking worlds that is closely related to, 
but has a little more structure than the orderings considered by Boutilier (as well as 
Grove and Katsuno and Mendelzon). The extra structure makes it easier to define a 
notion of conditioning. 

Given a formula cp, let nl^p) = mm{K,{w) : w \= </?}; we define K{false) = oo. We say 
that ip is believed with firmness a > in OCF k if K{ip) = and = Thus, ip is 

believed with firmness a if is unsurprising and the least surprising world satisfying -up 
has rank a. We define Bel(fi;) to consist of all formulas that are believed with firmness 
at least 1. 

Spohn defined a notion of conditioning on OCFs. Given an OCF n, a formula ip such 
that K{{p) < oo, and a > 0, K^^a is the unique OCF satisfying the property desired by 
Darwiche and Pearl — namely, if w and w' both satisfy (p or both satisfy -up, then revision 

^Spohn allowed ranks to be arbitrary ordinals, not just natural numbers, and did not allow a rank 
of oo, since, for philosophical reasons, he did not want to allow a world to be considered impossible. As 
we shall see, there are technical advantages to introducing a rank of oo. 
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by p should not change the relative rank of w and w', that is, k^,q,(w) — ■ 
k[w) — K,{w') — such that (f is believed with firmness a in k^^q- It is defined as follows: 



k{w) — K{ip) if w satisfies ip 

k{w) — k(~iv^) + a if w satisfies -^ip. 



Notice that K^^a is defined only if K{ip) < oo, that is, if p is considered possible. 
Darwiche and Pearl defined the following revision function on OCFs: 

{K if k{-^lp) > 1 
n^^i otherwise. 

Thus, if if is already believed with firmness at least 1 in k, then k is unaffected by a 
revision by otherwise, the effect of revision is to modify k by conditioning so that p 
ends up being believed with degree of firmness 1. Intuitively, this means that if p is not 
believed in k, in k * y9 it is believed, but with the minimal degree of firmness. 

It is not hard to show that if we take an agent's epistemic state to be represented by 
an OCF, then Darwiche and Pearl's semantics satisfies all the AGM postulates modified 
to apply to epistemic states (that is, Rl'-R8' in Section ||), except that revising by false is 
disallowed, just as in Boutilier's approach, so that R5' holds vacuously; in addition, this 
semantics satisfies Darwiche and Pearl's C1-C4, modified to apply to epistemic states. 
For example, C2 becomes 

C2'. IfipV- -^ij, then Bel((E * ^) * p) = Be\{E * p). 

Indeed, as Darwiche and Pearl observe, Boutilier's revision operator also satisfies Cl'- 
C4'; however, it has properties that they view as undesirable. Thus, Darwiche and Pearl's 
claim that their postulates are consistent with AGM is correct, if we think at the level 
of general epistemic states. On the other hand, Freund and Lehmann are quite right 
that R1-R8 and C1-C4 are incompatible; indeed, as they point out, R1-R4 and C2 are 
incompatible. The importance of making clear exactly whether we are considering the 
postulates with respect to the OCF k, or the belief set Bel(K) is particularly apparent 
here. I 

The fact that Boutilier's revision operator also satisfies Cl'-C4' clearly shows that 
these postulates do not capture all of Darwiche and Pearl's intuitions. Their semantics 
embodies further assumptions. Some of them seem ad hoc. Why is it reasonable to 
believe ip with a minimal degree of firmness after revising by p7 Rather than trying to 
come up with an improved collection of postulates (which Darwiche and Pearl themselves 
suggest might be a difficult task), it seems to us that a more promising approach is to 
find an appropriate ontology. 



^We note that in a recent version of their paper, Darwiche and Pearl |1997] use a similar technique 
to deal with the inconsistency of C2. 
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3.4 Lehmann's Revised Approach 



Finally, we consider Lehmann's "revised" approach to belief revision ||1995|] . With each 



sequence a of observations, Lehmann associates a belief set that we denote Bel((j). Intu- 
itively, we can think of Bel(cr) as describing the agent's beliefs after making the sequence 
a of observations, starting from her initial epistemic state. Lehmann allows all possible 
sequences of consistent formulas. Thus, he assumes that the agent does not observe false. 
We view Lehmann's approach essentially as taking the agent's epistemic state to be the 
sequence of observations made, with the obvious revision operator that concatenate a 
new observation to the current epistemic state. The properties of belief change depend 
on the function Bel. Lehmann requires Bel to satisfy the following postulates (where a 
and p denote sequences of formulas, and ■ is the concatenation operator): 

11. Bel(cr) is a consistent belief set 

12. If G Bel(o- • (f) 

13. l^^p e Bel((T ■ (p), then (p ^ e Bel(a) 

14. If G Bel(a), then Bel(a ■ ^ ■ p) = Be\{a ■ p) 

15. If h then Bel(cr ■ ip ■ ip ■ p) = Bel(o" ■ ip ■ p) 

16. If ^Tp ^ Bel(o" ■ if), then Bel(cr ■ ip ■ ip ■ p) = Bel(o" ■ (p ■ (p /\ip ■ p) 

17. Bel(a ■ -y? ■ y?) C Cl(Bel(a) U 



We refer the interested reader to [[Lehmann 1995[ | for the motivation for these postulates 



As Lehmann argues, the spirit of the original AGM postulates is captured by these pos- 
tulates. Lehmann views 15 and 17 as two main additions to the basic AGM postulates. 
He states that "Since postulates 15 and 17 seem secure, i.e., difficult to reject, the pos- 
tulates 11-17 may probably be considered as a reasonable formalization of the intuitions 
of AGM" ||Lehmann 1995| , Section 5]. Our view is that it is impossible to decide whether 



to accept or reject postulates such as 15 or 17 (or, for that matter, any of the other pos- 
tulates) without an explicit ontology. There may be ontologies for which 15 and 17 are 
reasonable, and others for which they are not. "Reasonableness" is not an independently 
defined notion; it depends on the ontology. The ontology of the next section emphasizes 
this point. 



4 Taking Observations to be Knowledge 

We now consider an ontology where observations are taken to be knowledge.^ 



^We remark that Rott [ 1991 , Section 6] outlines an approach where obervations are treated as knowl- 
edge, but does not provide an underlying ontology. The high-level intuition behind his approach is 
similar to the one we present here, although the details differ. 
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As we said in the introduction, in this ontology, the agent has some (closed) set of 
formulas that he knows to be true, which is included in a larger set of formulas that 
he believes to be true. The belief set can be viewed as the result of applying some 
nonmonotonic reasoning system grounded in the observations. We can think of there 
being an ordering on the strength of his beliefs, with the formulas known to be true — the 
observations and their consequences — having the greatest strength of belief. Because 
observations are taken to be knowledge, any formula observed is added to the stock of 
knowledge (and must be consistent with what was previously known). In this ontology, it 
is impossible to observe false. In fact, it is impossible to make any inconsistent sequence 
of observations. That is, if • • • , V^n is observed, then ipi A . . . A ipn must be consistent 
(although it may not be consistent with the agent's original beliefs). 

In earlier work [ I'riedman 1997 ; b'riedman and Halpern 1999|| , we presented one way 
of formalizing this ontology, based on the framework of Halpern and Fagin ||1989|| for 
modeling multi-agent systems (see [[bagin, Halpern, Moses, and Vardi 1995|| for more 
details). For modeling belief revision, we use this framework restricted to a single agent. 

The key assumption in this framework is that we can characterize the system by 
describing it in terms of a state that changes over time. Formally, we assume that at 
each point in time, the agent is in some local state. Intuitively, this local state encodes 
the information the agent has observed thus far. There is also an environment, whose 
state encodes relevant aspects of the system that are not part of the agent's local state. 
A global state is a tuple (se, Sa) consisting of the environment state Se and the local 
state Sa of the agent. A run of the system is a function from time (which, for ease of 
exposition, we assume ranges over the natural numbers) to global states. Thus, if r is a 
run, then r(0), r(l), ... is a sequence of global states that, roughly speaking, is a complete 
description of what happens over time in one possible execution of the system. We take 
a system to consist of a set of runs. Intuitively, these runs describe all the possible 
behaviors of the system, that is, all the possible sequences of events that could occur in 
the system over time. 

Given a system TZ, we refer to a pair (r, m) consisting of a run r E TZ and a time 
m as a point. If r(m) = (se, Sq), we define ra{m) = Sa and re{m) = Sg. We say two 
points {r,m) and {r',m') are indistinguishable to the agent, and write {r,m) ~(j {r',m'), 
if ra{m) = r'^{m'), i.e., if the agent has the same local state at both points. Finally, an 
interpreted system is a tuple (71,71), consisting of a system TZ together with a mapping 
TT that associates with each point a truth assignment to the primitive propositions. 

To capture the AGM framework, we consider a special class of interpreted systems: 
We fix a propositional language C We assume that the agent makes observations, which 
are characterized by formulas in C, and that her local state consists of the sequence of 
observations that she has made. We assume that the environment's local state describes 
which formulas are actually true in the world, so that it is a truth assignment to the 

""^^ Although the muhi-agent aspects of this framework does not play a role in our analysis here, we 
note that it allows for a natural extension of our results to multi-agent belief revision. This, however, is 
beyond the scope of this paper. 
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formulas in L. As observed by Katsuno and Mendelzon [ 1991a ], the AGM postulates 
assume that the world is static; to capture this, we assume that the environment state 
does not change over time. Formally, we are interested in the unique interpreted system 
Ij^AGM ^ tt) that consists of all runs satisfying the following two assumptions for every 
point (r, m): 

• The environment's state re(m) is a truth assignment to the formulas in C that 
agrees with vr at (r, m) (that is, 7r(r, m) = re(m)), and re{m) = re(0). 

• The agent's state ra(m) is a sequence of the form {ipi, . . . , ipm), such that ipiA. . -Aipm 
is true according to the truth assignment rf>{m) and ra{m — 1) = {ipi, . . . , ipm-i)- 

Notice that the form of the agent's state makes explicit an important implicit assumption: 
that the agent remembers all her previous observations. 

In an interpreted system, we can talk about an agent's knowledge: the agent knows 
ip at a. point (r, m) if ip holds in all points {r',m') such that {r,m) ~a {r',m'). It is easy 
to see that, according to this definition, if ra(m) = {ipi, . . . , ipm), then the agent knows 
If I A ... A !frn at the point {r,m): the agent's observations are known to be true in this 
approach. We are interested in talking about the agent's beliefs as well as her knowledge. 
To allow this, we added a notion of plausibility to interpreted systems in [[Friedman and 



Halpern 1997]] . We consider a variant of this approach here, using OCFs, since it makes 
it easier to relate our observations to Darwiche and Pearl's framework. 

We assume that we start with an OCF k, on runs such that n{r) ^ oo for any run 
r. Intuitively, k represents our prior ranking on runs. Initially, no runs is viewed as 
impossible. We then associate, with each point (r, m), an OCF k^'"''") on the runs. We 
define n^^'"^^ by induction on m. We take k^^'^^ = k, and we take = ^fj'™'' ^, 

where ra{m + 1) = {cpi, . . . , cpm+i)- Thus, k^^'"^^^^ is the result of conditioning k^^'"^^ on 
the last observation the agent made, giving it degree of firmness oo. Thus, the agent is 
treating the observations as knowledge in a manner compatible with the semantics for 
knowledge in interpreted systems. Moreover, since observations are known, they are also 
believed. 

Note that we could have described the same belief change process by considering 
an OCF on formulas, rather than runs. However, we feel that using runs and systems 
provides a better model of what is going on, and gives a more explicit ontology. We can 
then derive an OCF on formulas from the OCF on runs that we consider. 

As we show in [ [briedman 1997| ; [I'riedman and Halpern 1999|[ , this framework satisfies 
the AGM postulates Rl'-R8', interpreted on epistemic states. (Here we take the agent's 
epistemic state at the point (r, m) to consist of ra{m) together with k'^'^'^\) Moreover, 
we show this framework satisfies an additional postulate, which we call R9': 

(R9') If \/c -^{^ A t/j) then Be\{E *(f*^) = Be\{E *^A^). 
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This postulate captures the intuition that observations are taken to be knowledge, and 
thus observing ip and then ip is equivalent to observing if A ip. In fact, we show there 
that Rl'-R9' characterize, in a precise sense, revision in this framework. 

It is easy to verify that the framework also satisfies Darwiche and Pearl's postulates 
(appropriately modified to apply to epistemic states), except that the contentious C2 is 
now vacuous, since it is illegal to revise by ip and then hj ip if ip\ — iip. 

How does this framework compare to Lehmann's? Like Lehmann's, there is an explicit 
attempt to associate beliefs with a sequence of revisions. However, we have restricted 
the sequence of revisions, since we are treating observations as knowledge. It is easy 
to see that 11-13 and 15-17 hold in our framework. However, since we have restricted 
the sequence of observations allowed, some of these postulates are much weaker in our 
framework than in Lehmann's. In particular, 17 is satisfied vacuously, since we do not 
allow a sequence of the form a ■ -up ■ ip. On the other hand, 14 is not satisfied in our 
framework. Our discussion in the introduction suggests a counterexample. Suppose that 
initially, k{p A g) = 0, K(-ip A g) = 1, k{p A -ig) = 2, and k{^p A -ig) = 3. Thus, initially 
the agent believes both p and g, but believes p with firmness 1 and g with firmness 2. If 
the agent then observes -ip V -ig, he will then believe g but not p. On the other hand, 
suppose the agent first observes p. He still believes both p and g, of course, but now p 
is believed with firmness oo. That means if he then observes -ip V -ig, he will believe p, 
but not g, violating 14. However, a weaker variant of 14 does hold in our system: if the 
agent knows (p, then observing ip will not change her future beliefs. 

Did we really need all the machinery of runs and systems here? While we could 
no doubt get away without it, we believe that modeling time and the agent's state 
explicitly adds a great deal. In particular, by including time, we capture the belief revision 
process within the model. By including the agent's state and the environment, we can 
capture various assumptions about the agent's observational capabilities (this is especially 
relevant once we allow inaccurate observations) and generalize to allow multiple agents. 
See Poutilier, Friedman, and Halpern 1998| ; [Friedman and Halpern 1999| ; [Friedman 
and Halpern 199"^ for further discussion and demonstration of the advantages of this 
framework. 



5 Discussion 

The goal of this paper was to highlight what we see as some methodological problems 
in much of the literature on belief revision. There has been (in our opinion) too much 
attention paid to postulates, and not enough to the underlying ontology. An ontology 
must make clear what the agent's epistemic state is, what types of observations the agent 
can make, the status of observations, and how the agent goes about revising the epistemic 
state. 

We have (deliberately) not been very precise about what counts as an ontology, and 
clearly there are different levels of detail that one can provide. Although some papers 
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on belief revision have attempted to provide something in the way of an ontology, the 
ontology has typically been insufficiently detailed to verify the reasonableness of the 
postulates. For example, Gardenfors ||1988|| — whose ontology is far better developed 



than most — takes belief sets to consist of formulas that are accepted, and revision to be 
by formulas that are accepted. As we have seen, unless belief sets or the revision operator 
contain additional information (such as epistemic importance or strengths of beliefs) this 
ontology will violate Rl. On the other hand, unless we are careful about how we add such 
information, we may well violate some other axioms (such as R3 or R5). In any case, it 
is the job of the ontology to make completely clear such issues as whether observations 
are believed to be true or known to be true, and if they are believed, what the strength 
of belief is. This issue is particularly important if we have epistemic states like rankings 
that are richer than belief sets. If observations are believed, but not necessarily known, to 
be true, then it is not clear how to go about revising such a richer epistemic state. With 
what degree of firmness should the new belief be held? No particular answer seems to us 
that well motivated. It may be appropriate for the user to attach degrees of firmness to 
observations, as was done in fGoldszmidt 199^ ; [Williams 1994| ; [Wobcke 1995| | (following 
the lead of Spohn ||1988|| ); we can even generalize to allowing uncertain observations 
Dubois and Frade 1994 1- 



We are currently interested in finding alternative ontologies where observations are not 
taken to be knowledge. One such ontology, which is motivated by ideas from stochastic 
processes, is described in fBoutilier, Friedman, and Halpern 1998|| . This ontology explic- 
itly models how "noisy" observations come about. Roughly speaking, we assign a prior 
plausibility to observing a formula in a world w. When we observe ip we update our 
plausibility measure on worlds by considering the plausibility of observing in each of 
these worlds. Of course, we can then consider various assumptions on this prior plausi- 
bility. For example, we might say that we believe observations to be true. That is, our 
prior plausibility of observing in worlds where it is true is higher than the plausibility 
of observing Lp in worlds where it false. We note that this assumption does not imply 
that if G Bel(if * if), since we have to combine the plausibility of observing Lp with the 
plausibility of if before the observation was made. If (p is considered implausible in K, 
then the single observation of p might not suffice to make (p more plausible than -^p. 
(For example, the medieval scientist probably would not change her beliefs about the 
speed of falling objects after a single experiment.) 

In this paper we have focused on belief revision. However, the need to clarify the 
underlying ontology goes far beyond belief revision. Much the same comments can be 
made for all the work on nonmonotonic logic as well (this point is essentially made in 
Halpern 1993|| ). Not surprisingly, our critique applies to other approaches belief change 



as well, and in particular to Katsuno and Mendelzon's belief update [p.991a|| . Although 



the motivation described by Katsuno and Mendelzon is different than that of revision, 
the discussion of update is stated in terms of postulates about belief sets. Thus, for 
example, the distinction between the agent's belief set and epistemic state arises in 
update as well: update is defined as function from (belief sets x formulas) to belief sets. 
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However, the agent's belief set does not uniquely determine the outcome of update. This 
is demonstrated, for example, by Katsuno and Mendelzon's semantic characterization 
that requires the agent to have a ternary relation on possible worlds such that Wi <^ W2 
holds if the agent considers wi to be "closer" to w than W2- Other issues we raise here 
also apply to update for similar reasons. 

The ontology we propose in Section ^, where observations are treated as knowledge, 
can be applied to update as well. Moreover, the assumption that observations are true is 
less problematic for iterated update: since update does not assume that propositions are 
static, we can consider runs where cp is true at one time point, and false at the next one. 
Thus, assuming that observations are known to be true does not rule out sequences of 
observations of the form ip, -up, .... In fact, all sequences of consistent observations are 
allowed. We refer the reader to [ Friedman 1997 ; Friedman and Halpern 1999 1 for more 
details. 

It seems to us that many of the intuitions that researchers in the area have are moti- 
vated by thinking in terms of observations as known, even if this is not always reflected 
in the postulates considered. We have examined carefully one particular instantiation of 
this ontology, that of treating observations as knowledge. We have shown that, in this 
ontology, some postulates that seem reasonable, such as Lehmann's 14, do not hold. We 
do not mean to suggest that 14 is "wrong" (whatever that might mean in this context). 
Rather, it shows that we cannot blithely accept postulates without making the underly- 
ing ontology clear. We would encourage the investigation of other ontologies for belief 
change. 
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